A Socratic Approach To Interviewing

The “socratic approach” to teaching is to ask students pointed questions to lead students to discover things for themselves by thinking about the question and its possible answers. Could we adopt this approach to screening technical candidates? If so would it help us to better discover good technical candidates?

Socrates And The Socratic Method

Most people have run across the name “Socrates” at some point in their lives. He was so central to the development of classical Greek philosophy that historians of philosophy divide it into “pre-Socratic” and “post-Socratic.” Not bad for a person who apparently never wrote down one of his lessons. It’s true; all we know about Socrates comes from his best student, Plato, and a bit from some other Athenian writers who lived around the same time. Socrates himself never wrote anything. Or if he did none of it has survived to our time.

One interesting idea attributed to him is the notion of the Socratic Method. The Socratic Method consists of asking questions and then asking follow up questions based on the answers given. For example we might start with “Is it ethical to cheat a wealthy person on a land deal?” and you might respond, “No, cheating another person is never ethical” and I may respond “But it’s very likely they acquired some of their wealth via unethical means—does that have any bearing on your answer?” and so forth and so on. It’s not that you’ll ever reach a final indisputable answer; it’s the process of examining the questions and their responses. This is called the “Socratic method”—asking open-ended questions and following up on the answers from the respondent with maybe more open-ended questions.

The Socratic Method Of Interviewing

Now can we apply this to the issue of interviewing technical candidates? I think we can. Granted there are some questions that have a definite answer, there are far more questions that require a chain of thought to find a good answer. And the answers are often not definitely right or wrong; they’re simply more or less useful in the particular context.

Let’s say for example I’m screening a candidate for a software engineer job. I could ask them to code things for me (and that seems like a good approach) but I have found in the past that asking open-ended questions and then following up on them seems to help me determine what someone is capable of as a developer.

For example, I sometimes start with the question: “You’ve been given a task to estimate. You’re very confident that the task in question will take 3 weeks to complete and you tell your boss that. Your boss says—‘Uh, we need it in a week.’ What do you say to your boss in response?” There is no right or wrong answer here. When I ask this question I’m trying to gauge how the candidate thinks. Below are some responses I’ve gotten and what I think of them:

“I work harder and get it done in a week”

is my least acceptable answer because not only is it unrealistic it’s also encouraging the manager/management to think that no matter what we, as developers, say they can simply demand things faster and we’ll comply.

Potential Follow-Up Questions:

“You’ve estimated three weeks. How are you planning to work harder to compress a three week job into a one week job?”

“Do you doubt your ability to estimate jobs?”

“Is pushing back on the one week deadline not an option? If not, why isn’t it an option?”

“I lie and say I can get it done in a week”

again, not a great response. You don’t want people willing to lie on your team. To be frank I only heard this response once; I think he was actually trying to impress me with his candor.

Potential Follow-Up Questions:

“What happens at the end of the week when the job isn’t done?”

“Are you comfortable with telling people what they want to hear regardless of the truth?”

“I speak to my manager about what can be cut out of the job so we can get it done in a week”

This is the response I’m looking for. The business has needs but we don’t want to ship hacky crap just to meet a deadline. So any sort of compromise is probably for the best both ways.

Potential Follow-Up Questions:

“Assuming they’re amenable to cutting out work, how do you ensure the necessary work gets done later?”

“What would you consider essential and what would you consider ‘nice to have’?”

“How do you ensure you meet your one week deadline now that you’ve negotiated less work?”

I hope the above examples help to illustrate my point.

Possible Benefits of Socratic Interviewing

These sort of open-ended questions have a few benefits:

1. They may lead to unscripted discoveries. That is they may lead you and the candidate to discuss things neither of you had previously considered but which are important.
2. Open-ended questions are harder to feed to an LLM. In my experience anyway, a candidate could feed open-ended questions into an LLM and get a reasonable response but since the question is less specific the answer will likewise be less specific and hence may help you to flush out a poser.